Papers with cross-lingual alignment
CLEAR: Cross-Lingual Enhancement in Retrieval via Reverse-training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual embedding models often struggle to capture cross-lingual alignment during training. |
| Approach: | They propose a novel loss function that leverages an English passage as a bridge to strengthen alignments between target language and English. |
| Outcome: | The proposed model improves retrieval performance across cross-lingual scenarios while minimizing performance degradation in English. |
A Novel Paradigm Boosting Translation Capabilities of Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on LLMs focused on supervised fine-tuning but their effectiveness has been limited. |
| Approach: | They propose a paradigm consisting of three stages: Secondary Pre-training using extensive monolingual data, Continual Pre- training with interlinear text format documents, and Leveraging source-language consistent instruction for supervised fine-tuning. |
| Outcome: | The proposed approach surpasses previous work and achieves superior performance compared to models such as NLLB-54B(CITATION) and GPT3.5-text-davinci-003. |
SkipCLM: Enhancing Crosslingual Alignment of Decoder Transformer Models via Contrastive Learning and Skip Connection (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing contrastive learning methods for cross-lingual alignment are not effective for multilingual machine translation tasks. |
| Approach: | They propose a method that augments contrastive learning for cross-lingual alignment with a trainable skip connection to preserve information crucial for accurate target language generation. |
| Outcome: | Experiments with XGLM-564M on the Flores-101 benchmark show that the proposed method preserves crucial information crucial for accurate target language generation. |
The Impact of Positional Encodings on Multilingual Compression (2021.emnlp-main)
Copied to clipboard
| Challenge: | Several modifications have been proposed to improve monolingual language models, but none of them result in better multilingual models. |
| Approach: | They propose to add positional encodings to token embeddings to preserve word-order information in a non-autoregressive setting. |
| Outcome: | The proposed modifications tend to improve monolingual models, but none improve multilingual models. |
Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Models (2025.acl-long)
Copied to clipboard
Anirudh Sundar, Sinead Williamson, Katherine Metcalf, Barry-John Theobald, Skyler Seto, Masha Fedzechkina
| Challenge: | Large language models (LLMs) exhibit impressive performance on a variety of tasks from text summarization to zero-shot common-sense reasoning. |
| Approach: | They propose to manipulate the embedding space of mLLMs by manipulating its activations to steer generation into the desired direction. |
| Outcome: | The proposed model interventions improves alignment of cross-lingual representations in multilingual large language models with up to 2x improvements in top-1 accuracy on cross-linguistic retrieval tasks. |
When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Large Language Models process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. |
| Approach: | They investigate the development of language-agnostic concept spaces during pretraining of EuroLLM using the causal interpretability method of activation patching. |
| Outcome: | The proposed model is language-agnostic and enables cross-lingual transfer . the model is able to process multilingual inputs, but lacks cross-linguistic alignment . |
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024.findings-acl)
Copied to clipboard
Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, Daniel Khashabi
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
COMMIT: Code-Mixing English-Centric Large Language Model for Multilingual Instruction Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | COMMIT improves the exact match score of low-resource language QA by 32x. |
| Approach: | They propose to specialize instruction tuning to deviate from English-centric instruction tuning . they propose to perform cross-lingual alignment to overcome data imbalance . |
| Outcome: | The proposed method improves the exact match score of low-resource language QA by 32x. |
AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities, but performance and cross-lingual alignment often lag for non-dominant languages. |
| Approach: | They propose a representation-level framework to enhance multilingual performance of pre-trained LLMs by integrating multilingual semantic alignment and language feature integration. |
| Outcome: | The proposed framework improves multilingual capability of pre-trained LLMs by bringing representations closer and improving cross-lingual alignment. |
Multi-Hall-SA: A Cross-lingual Benchmark for Multi-Type Hallucination Detection in Low-Resource South African Languages (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models generate false or unsupported information, which can be difficult to detect in low-resource languages. |
| Approach: | They propose a cross-lingual benchmark for hallucination detection spanning English and South African languages. |
| Outcome: | The proposed model detects 23.6% fewer hallucinations in South African languages compared to English . human validation confirms the quality and cross-lingual alignment of the model . |
Unveiling Linguistic Regions in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on how LLMs achieve cross-lingual alignment and generalization have not explored the intrinsic mechanisms of how they achieve crosslingual alignment. |
| Approach: | They propose to remove a core region that corresponds to linguistic competence and set parameters to zero to reduce performance across 30 different languages. |
| Outcome: | The proposed model can be used to perform tasks requiring abstract knowledge and reasoning in complex languages. |
Exploring Alignment in Shared Cross-lingual Spaces (2024.acl-long)
Copied to clipboard
| Challenge: | a new study examines the degree of alignment between languages in multilingual embeddings . cross-lingual embeds are designed to encode linguistic concepts that bridge equivalent semantic meaning . a comprehensive approach is needed to address these questions. |
| Approach: | They employ clustering to uncover latent concepts within multilingual models . they introduce two metrics to quantify alignment and overlap of these concepts . |
| Outcome: | The proposed model can capture linguistic nuances across languages, but is not language-agnostic? the proposed model is able to capture nuances in multiple languages, the authors say. |
PPT: A Minor Language News Recommendation Model via Cross-Lingual Preference Pattern Transfer (2025.acl-long)
Copied to clipboard
| Challenge: | Existing systems for minor language news recommendation lack interaction with content . linguistic gap can lead to inaccurate modeling of minor language content despite strong English capability . |
| Approach: | They propose a minor language news recommendation model by cross-lingual preference pattern transfer . their model employs the widely used two-tower architecture and large language model as the backbone of the news encoder . |
| Outcome: | The proposed model outperforms existing models on 15 minor languages. |
Multilingual Generative Retrieval via Cross-lingual Semantic Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for multilingual retrieval still face cross-lingual identifier misalignment and identifiere inflation. |
| Approach: | They propose a framework that unifies semantically equivalent multilingual keywords into shared atoms to align semantics and compresses the identifier space. |
| Outcome: | The proposed framework improves cross-lingual alignment and reduces redundancy. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent mPLMs have shown impressive performance on crosslingual transfer tasks . however, the performance is often hindered when a lowresource target language is written in a different script than the high-resource source language. |
| Approach: | They propose a transliteration-based method to improve cross-lingual alignment between languages using diverse scripts. |
| Outcome: | The proposed method outperforms the original model on Englishcentric transfer tasks up to 50%. |
Cross-lingual Aspect-based Sentiment Analysis with Aspect Term Code-Switching (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on Aspect-based sentiment analysis (ABSA) focus on English texts, but handling it in resource-poor languages remains a challenge. |
| Approach: | They propose an unsupervised cross-lingual transfer method for the Aspect-based sentiment analysis task . they propose an aspect code-switching mechanism to augment training data with code-linked bilingual sentences . |
| Outcome: | The proposed method preserves task-specific knowledge in the target language. |
KnowCoder-X: Boosting Multilingual Information Extraction via Code (2025.findings-acl)
Copied to clipboard
Yuxin Zuo, Wenxuan Jiang, Wenxuan Liu, Zixuan Li, Long Bai, Hanbin Wang, Yutao Zeng, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
| Challenge: | Empirical evidence indicates that Large Language Models exhibit spontaneous cross-lingual alignment in Information Extraction (IE) however, a significant imbalance across languages persists, highlighting an underlying deficiency. |
| Approach: | They propose a code LLM with advanced cross-lingual and multilingual capabilities for universal IE that standardizes the representation of multilingual schemas using Python classes and conducts IE alignment instruction tuning on translated instance prediction task. |
| Outcome: | The proposed model surpasses ChatGPT and SoTA by 30.17% without training in 29 unseen languages and significantly improves cross-lingual IE transferability. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |
Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear. |
| Approach: | They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario . |
| Outcome: | The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training . |
The Emergence of Semantic Units in Massively Multilingual Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Massively multilingual models can process text in several languages relying on a shared set of parameters, but little is known about the encoding of multilingual information in single network units. |
| Approach: | They propose to use a shared set of parameters to encode multilingual information in single network units. |
| Outcome: | The proposed model achieves higher scores in semantic encoding in languages with more cross-lingual alignment than those with more shared cross-linguistic substrate. |
From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing alignment benchmarks focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. |
| Approach: | They propose a novel cross-lingual alignment evaluation method based on the consistency of parallel sentences to assess model alignment. |
| Outcome: | The proposed method achieves a correlation of 0.9556 with downstream tasks performance and 0.8524 with transferability even with a small dataset. |
SiLP: Enhancing Non-Dominant Language Capabilities with a Selective Bidirectional Language Projection Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of large language models rely on additional training objectives or language-specific parameters. |
| Approach: | They propose a bidirectional language projection framework that enables efficient multilingual alignment and language shift using the intrinsic parameters. |
| Outcome: | The proposed framework improves performance of non-dominant languages and improves internal representations. |